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Summary: 

The  Interactive  Multisensor  Analysis  Training  (IMAT)  project  is  aimed  at  improving  the  preparation  of 
operational  users  of  undersea-warfare  sensor  systems.  The  effort  has  focused  on  training  at  all  levels  from 
initial  individual  training  ashore  through  team,  platform,  and  collective  training  at-sea,  at  all  skill  levels  from 
apprentice  sensor  operators  to  senior  tactical  commanders.  Operators  and  tacticians  at  all  levels  need  a  deep 
and  scientifically  accurate,  but  not  necessarily  formal,  understanding  of  the  physical  principles  that  underlie 
tactical  employment  of  their  sensors.  IMAT  systems  use  model-based  scientific  visualizations,  including 
three-dimensional  graphics  and  animations,  to  illustrate  complex  physical  interactions  in  mission-relevant 
contexts,  and  to  provide  interactive  virtual  laboratories  in  which  the  principles  can  be  explored.  Concepts  in 
instruction  include  radiated  acoustic  characteristics,  propagation  in  range-dependent  environments,  and 
sensor  properties.  Training  systems  provide  exploratory  environments  in  which  operators  and  tacticians  can 
examine  the  effects  of  change  in  any  of  the  variables  involved  in  the  end-to-end  sequence  of  emission, 
transmission,  reflection  and  detection.  Sensor  settings,  environmental  conditions,  and  target  characteristics 
can  all  be  modified  through  a  "what-if  simulation  approach.  These  technologies  have  been  applied 
effectively  in  basic  and  advanced  sensor  operations/employment  courses;  in  individual  and  team  training 
simulators,  and  in  on-board  training.  At  the  battle-group  and  theater  level,  new-technology  systems  are  used 
for  decision  support  during  at-sea  exercises  and  operations,  and  for  post-event  reconstruction  and 
performance  analysis.  This  paper  describes  the  IMAT  training  philosophy  and  approach,  the  design  of 
training  systems,  and  training  effectiveness. 


Introduction 

The  Interactive  Multisensor  Analysis  Training  (IMAT)  program  is  a  major  visualization  effort  in  the  US 
Navy  conducted  over  the  past  nine  years.  The  objective  in  this  program  is  to  provide  performance  support 
and  training  systems  for  extremely  difficult  cognitive  tasks  involved  in  Anti-Submarine  Warfare  (ASW). 
The  effort  has  focused  on  training  at  all  levels  from  initial  training  ashore  through  team,  platform,  and 
collective  training  at-sea,  at  all  skill  levels  from  apprentice  sensor  operators  to  senior  tactical  commanders. 

ASW  involves  the  use  of  a  variety  of  sensor  systems  to  locate  an  opposing  threat  or  target  submarine  in  the 
ocean  environment,  often  while  avoiding  counterdetection.  In  particular: 

Threat  energy  Sources  refer  to  the  acoustic,  electromagnetic,  or  electro-optical  emission  and/or 
reflection  properties  associated  with  undersea  objects.  This  includes  the  types  and  parameters  of 
signals  emitted  by  them,  such  as  frequency  and  amplitude.  It  also  includes  the  azimuthal  or  “aspect 
dependent”  variability  associated  with  signals  in  the  case  of  “passive”  systems,  and/or  the  reflective 
properties  and  aspect  dependencies  in  the  case  of  “active”  systems.  A  source  that  is  sufficiently 
energetic  to  be  remotely  detectable  constitutes  a  vulnerability. 

Environment  refers  to  the  transmissive  properties  of  the  media  through  which  (acoustic  or 
electromagnetic)  energy  is  propagated.  The  environment  affects  the  path  which  energy  takes  as  it  is 

Paper  presented  at  the  RTO  HFM  Symposium  on  “Advanced  Technologies  for  Military  Training", 
held  Genoa,  ItalylS  -  15  October  2003,  and  published  in  RTO-MP-HFM-101. 


RTO-MP-HFM-101:  Paper  4 


Page  1  of  14 


Report  Documentation  Page 

Form  Approved 

0MB  No.  0704-0188 

Public  reporting  burden  for  the  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering  and 
maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information, 
including  suggestions  for  reducing  this  burden,  to  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports,  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington 

VA  22202-4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  a  penalty  for  failing  to  comply  with  a  collection  of  information  if  it 
does  not  display  a  currently  valid  0MB  control  number. 

1.  REPORT  DATE 

00  APR  2004 

2.  REPORT  TYPE 

N/A 

3.  DATES  COVERED 

4.  TITLE  AND  SUBTITLE 

Interactive  Multisensor  Analysis  Training 

5a.  CONTRACT  NUMBER 

5b.  GRANT  NUMBER 

5c.  PROGRAM  ELEMENT  NUMBER 

6.  AUTHOR(S) 

5d.  PROJECT  NUMBER 

5e.  TASK  NUMBER 

5f.  WORK  UNIT  NUMBER 

7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES) 

Space  and  Naval  Warfare  Systems  Center,  Code  2302  53560  Hull  St.  Bldg 
586  San  Diego,  CA  92152  USA;  Office  of  Naval  Research  Code  34  CM 

FNC  800  N.  Quincy  St.  Arlington,  VA  22217  USA 

8.  PERFORMING  ORGANIZATION 

REPORT  NUMBER 

9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES) 

10.  SPONSOR/MONITOR’S  ACRONYM(S) 

11.  SPONSOR/MONITOR’S  REPORT 
NUMBER(S) 

12.  DISTRIBUTION/AVAILABILITY  STATEMENT 

Approved  for  public  release,  distribution  unlimited 

13.  SUPPLEMENTARY  NOTES 

See  also  ADM001667,  NATO  RTO-MP-HFM-101  Advanced  Technologies  for  Military  Training 
(Technologies  avancees  pour  lentrainement  militaire)..  The  original  document  contains  color  images. 

14.  ABSTRACT 

15.  SUBJECT  TERMS 

16.  SECURITY  CLASSIFICATION  OF: 

17.  LIMITATION  OF 
ABSTRACT 

uu 

18.  NUMBER 
OF  PAGES 

14 

19a.  NAME  OF 
RESPONSIBLE  PERSON 

a.  REPORT 

unclassified 

b.  ABSTRACT 

unclassified 

c.  THIS  PAGE 

unclassified 

standard  Form  298  (Rev.  8-98) 

Prescribed  by  ANSI  Std  Z39-18 


nROANIZATfON 


reflected  and  refracted,  and  the  amount  of  loss  that  occurs  as  a  result  of  transmission  through  the 
environment.  In  the  passive  case,  one-way  paths  and  path  loss  are  studied,  while  in  the  active  case 
both  outgoing  and  reflected  transmission  is  involved.  Environmental  properties  that  affect 
transmission  include  the  refractivity  of  the  medium,  ambient  or  other  interfering  noise  in  the 
environment  that  affects  signal-to-noise  ratio,  directional  properties  of  noise,  ocean  bottom  or  terrain 
absorption  and  contour,  reverberation,  and  other  factors.  Path  loss  is  also  a  function  of  frequency 
(and  in  the  case  of  electromagnetic  energy  so  is  refraction). 

The  Sensor  or  energy  Receiver  and  its  associated  signal  Processor  determine  how  received 
vulnerabilities,  as  affected  by  the  environment,  are  sensed,  manipulated  and  displayed  for  human 
operators,  tacticians,  and  strategists.  These  systems  also  are  controlled  by  their  operators  who  are 
seeking  to  optimize  processing  so  that  highest  sensitivity  is  achieved. 

These  individual  topics  cannot  be  understood  in  isolation  from  one  another.  For  example,  sensor 
optimization  depends  on  knowledge  of  the  threat  vulnerabilities  and  the  intervening  environment.  A 
fundamental  training  problem  is  that  changes  in  any  of  the  source,  environment,  or  sensor  variables  interact 
with  all  the  others.  And,  for  multiplatform  operations  (those  involving  multiple  ships  and  aircraft),  these 
triangular  interactions  increase  exponentially. 

ASW  is  an  incredibly  complicated  task:  At  all  levels  from  individual  sensor  operator  to  senior  commander, 
mission  success  depends  on: 

•  Correctly  anticipating  the  relative  merits  of  position,  speed,  maneuver  and  sensor/weapon 
employment.... 

•  For  each  platform  in  the  battle  force... 

•  Against  the  possible  range  of  threat  options. . . 

•  Within  highly  variable  environmental  conditions,  . . . 

•  In  a  rapidly  changing  combat  situation  in  which  intelligent  opponents  try  to  avoid  detection, 
confuse  identification  and  gain  tactical  advantage... 

•  Where  other  missions,  such  as  air  strike  or  missile  defense,  may  have  equal  or  greater  importance 
to  battle  force  survival. 

To  cope  with  tasks  of  this  complexity,  operators  and  tacticians  need  a  deep  understanding  of  the  principles 
underlying  operation  and  employment  of  their  sensors. 


Cognitive  Complexity 

Complex  tasks  like  those  involved  in  ASW  are  known  to  be  difficult  to  learn.  Nearly  50  years  ago,  Piaget 
noted  the  difficulty  of  tasks  that  require  coordinating  more  than  one  dimension  of  variation  (Inhelder  & 
Piaget,  1958).  Over  30  years  ago,  the  state  of  Massachusetts  (and  since  then  many  other  governments)  found 
it  necessary  to  impose  a  unit  pricing  law,  because  of  the  difficulty  most  people  have  in  determining  best 
value  among  differently  sized  and  priced  containers  of  the  same  product — a  relatively  straightforward  two- 
dimensional  task.  Feltovich  and  his  colleagues  (e.g.  Feltovich,  Spiro,  &  Coulsen,  1991)  have  described  some 
features  of  tasks  and/or  problems  that  make  them  difficult.  We  have  added  a  few  additional  criteria  in  the  list 
below: 

•  Abstract  (versus  concrete).  Physical  phenomena  are  invisible  and  the  underlying  cause  and  effect 
relationships  cannot  be  observed.  Examples  include  the  propagation  of  sound,  or  the  patterns  of 
sensitivity  of  an  acoustic  sensor. 
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•  Multi-variate  (versus  univariate):  Multiple  underlying  causes  can  affect  an  outcome.  For  example,  the 
refraction  of  sound  in  water  results  from  sound-speed  variation  with  depth,  which  in  turn  depends  on 
variations  among  salinity,  pressure,  and  temperature  with  depth. 

•  Interactive  (versus  separable  or  additive).  Underlying  causes  may  interact  with  each  other,  with 
outcomes  dependent  on  the  interaction  of  variables  in  addition  to  each  variable  acting  separately. 

•  Continuous  (versus  discrete).  The  dimensions  of  variation  are  continuous  For  example,  speed,  pressure, 
and  temperature  are  all  continuous  variables.  Rather  than  merely  memorizing  discrete  state  changes,  the 
learner  must  understand  the  effects  of  continuous  change. 

•  Non-linear  (versus  linear).  The  relationship  of  outcome  to  an  underlying  dimension  is  not  a  simple 
straight-line  function;  rather  relationships  may  be  exponential,  logarithmic,  or  even  more  complex.  For 
example,  energy  loss  in  propagation  often  involves  an  inverse-square  relationship. 

•  Dynamic  (versus  static).  The  process  of  variation  itself  is  the  subject  of  analysis,  rather  than  end  or 
intermediate  states.  A  few  frozen  moments  in  time  are  not  sufficient  to  characterize  the  underlying 
variation. 

•  Simultaneous  (versus  sequential).  Outcomes  vary  continuously  with  changes  in  underlying  variables, 
rather  than  as  a  succession  of  states. 

•  Conditional  (versus  universal).  Relationships  among  variables  and  outcomes  may  depend  on  particular 
boundary  conditions  or  other  contextual  events.  There  may  be  exceptions  to  general  rules  or  they  may 
apply  only  in  certain  circumstances  and  not  in  others. 

•  Uncertain  (versus  certain).  Exact  values  on  underlying  variables  may  not  be  known  precisely;  instead 
they  may  be  interpolations,  estimates,  or  approximations. 

•  Ambiguous  (versus  unique).  The  same  combination  of  circumstances  may  result  in  multiple  outcomes, 
or  the  same  outcome  may  be  the  result  of  different  combinations  of  circumstances. 

Antisubmarine  Warfare  tasks  involve  all  of  these  attributes.  For  example,  understanding  probability  of 
detection  involves  the  interaction  of  the  target’s  radiated  signals  (which  vary  in  three  spatial  dimensions 
around  the  target  by  frequency  by  time);  the  intervening  environment  which  may  distort  or  differentially 
enhance  or  attenuate  signals  at  particular  frequencies;  and  detectability  of  signals  by  a  particular  shipboard 
sensor  (with  directional  sensitivity  which  also  varies  in  frequency  and  in  three  spatial  dimensions).  This 
problem  is  further  complicated  when  such  factors  as  radiated  noise  variation  with  target  speed  and  depth, 
ownship  motion  effects  on  directional  frequency  response  of  sensors,  relative  motion  between  target  and 
sensor,  and  multipath  interactions  are  considered.  There  are  multiple,  continuous,  interactive,  nonlinear, 
highly-dynamic  dimensions  of  variability,  with  uncertainty  and  ambiguity,  throughout  the  ASW  problem 
space. 


Interactive  Multisensor  Analysis  Training  (IMAT) 

IMAT  is  designed  to  make  difficult  scientific  and  technical  concepts  underlying  sensor  employment  and 
tactical  planning  comprehensible  to  their  operational  users.  Operators,  tacticians  and  senior  decision  makers 
each  need  to  acquire  a  deep  and  scientifically  accurate,  but  not  necessarily  formal,  understanding  of  the 
physical  principles  that  underlie  tactical  sensor  employment  and  planning.  To  meet  this  requirement,  the 
IMAT  program  is 

•  developing  training  systems  which  integrate  computer  models  of  physical  phenomena  with  scientific 
visualization  technologies  to  demonstrate  the  interactive  relationships  of  threat,  environment,  and  sensor 
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for  operator  training,  and  interactions  of  multiple  sensor  systems  for  tactician  training  from  individual 
platform  to  battlegroup  to  theater; 

•  developing  training  and  performance  support  systems  using  modeling  and  visualization  technologies; 
and 

•  integrating  curricula  to  provide  training  on  sensor  employment  and  high-level  tactical  planning  skills. 

The  most  important  design  strategy  in  IMAT  is  to  use  computational  physics  models  as  the  basic 
representation  of  task  content.  Models  of  physical  phenomena  and  databases  of  environmental  observations 
are  integrated  with  scientific  visualization  technologies  to  build  systems  with  which  users  can  interact. 
These  systems  are  used  for  analysis  of  cognitive  requirements  underlying  successful  task  performance,  in 
two  ways.  First,  the  physical  relationships  underlying  sensor  employment  are  explicitly  identified,  and 
second,  task  experts  can  interact  with  the  systems  to  construct  detailed  representations  of  threat-sensor- 
environment  variables  and  relationships.  The  visualizations  can  then  be  used  in  training  episodes  that 
demonstrate  the  interactive  relationships  among  all  the  variables  involved.  Next,  the  modeling  and 
visualizations  are  used  during  exercises  and  deployments  to  identify  decision  requirements,  develop  and 
refine  user  interfaces,  and  build  tactical  decision  aids.  These  are  iteratively  refined  in  a  build-test-build 
approach.  Ultimately,  systems  are  evolved  which  provide  tactical  decision  aiding,  while  at  the  same  time 
allowing  drill-down  inspection  for  tactical  situation  analysis,  as  well  as  exploration  of  variability  in  (a) 
assumptions  about  presumed  target  behavior  and  vulnerability;  (b)  own-sensor  employment;  and,  (c) 
environmental  effects  on  sensor  effectiveness. 


Model-based  visualization: 

Scientific  Visualization  is  a  critically  important  enabling  technology  for  the  design  of  modem  mechanical 
systems,  in  medical  imaging,  and  in  most  of  the  physical  and  engineering  sciences.  There  are  thousands  of 
university  and  industrial  projects  in  which  visualization  is  employed  to  solve  engineering,  medical,  or  design 
projects.  Most  efforts  are  aimed  at  users  with  graduate  degrees  in  a  scientific  field,  and  only  a  few  involve 
training  applications.  A  recent  overview  of  information  visualization  is  given  by  Card,  Mackinlay  and 
Shneiderman  (1999).  (Note:  An  aimotated  bibliography  of  Scientific  Visualization  web  sites  is  located  at 
http://www.nas.nasa.gov/GroupsA^isTech/visWeblets.html). 

IMAT  uses  extensive  scientific  visualization  and  takes  advantage  of  work  in  a  large  number  of  university  and 
US  Navy  research  laboratories  to  develop  models  and  databases.  These  include  radiated  noise  models  and 
databases  which  describe  characteristics  of  sound  sources;  oceanographic  models  and  databases  which 
provide  high-resolution  bathymetric  and  bathythermographic  information,  ambient  noise,  bottom 
composition,  meteorological  and  other  physical  effects  on  propagation;  and  sensor  performance  models 
including  recent  developments  which  take  account  of  practical  (but  tactically  relevant)  effects  such  as  array 
motion.  In  general,  the  approach  has  been  to  adopt  these  from  their  controlling  organizations  who  are 
responsible  for  validation  and  verification.  IMAT  provides  the  visualization  tools  that  allow  the  operational 
and  tactical  implications  of  interationships  to  become  observable  rather  than  invisible. 

Sample  IMAT  visualizations  are  given  in  figures  1  through  3. 
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The  Acoustic  Source  Display  (Figure  1)  is  an  example  visualization  of  the  internal  components  of  a 
submarine.  This  is  one  view  into  a  modeled  acoustic  laboratory  for  sound  sources.  The  propulsion  and 
auxiliary  systems  depicted  in  the  diagram  can  all  be  animated  to  show  how  they  operate,  and  the  animations 
can  be  linked  to  recordings  of  acoustic  data  or  to  an  audio  simulation  to  show  how  acoustic  parameters  such 
as  frequency  are  related  to  the  physical  operation.  The  user  can  select  a  motor,  pump  or  other  object  in  the 
diagram,  which  will  highlight  it  and  display  a  textual  description  of  that  object.  Frequency  lines  associated 
with  the  component  are  highlighted  on  the  sound  spectrogram  in  the  bottom  part  of  the  display.  Each  object 
is  also  linked  to  a  more  detailed  three-dimensional  representation  that  enables  the  student  to  gain  a  better 
understanding  of  how  complex  assemblies  work,  why  they  generate  certain  signals,  and  how  signals  relate  to 
operating  mode  and  speed.  Modeled  objects  in  the  acoustic  laboratory  include  examples  of  diesel  engines, 
turbines,  reduction  gears,  pumps,  propellers,  motors,  generators,  compressors,  and  blowers.  Each  object  is 
also  linked  to  a  high  fidelity  acoustic  simulator  in  which  parameters  that  control  the  simulation  can  be  varied 
and  explored  for  instructional  purposes.  Motors,  pumps,  clutches,  and  other  components  can  be  activated 
and  deactivated  to  show  variations  in  operating  mode;  features  such  as  number  of  cylinders,  power  cycles,  or 
gear  ratios  can  be  changed;  and  depth  and  speed  changes  can  be  made,  with  all  changes  properly  reflected  in 
the  visual  and  auditory  displays. 
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Figure  2  is  a  view  of  an  interactive  model-based  animation  for  explaining  the  principles  of  phased 
array  beamforming.  Initially,  basic  concepts,  such  as  element  spacing  related  to  frequency,  and  in-phase  vs. 
out-of-phase  arrival,  are  introduced  using  a  simple  two-element  array.  Later  more  complicated  arrays  of 
acoustic  sensors  are  introduced.  In  Figure  2,  the  top  right  panel  shows  a  3-D  rendered  view  of  the  3dB-down 
isosensitivity  surface  for  sound  of  a  given  frequency  arriving  at  a  notional  10-element  line  array.  In  the 
simulation,  inter-element  spacing  and  phase  delays  are  all  adjustable.  The  top-center  panel  shows  delays 
applied  to  steer  sensitivity  for  a  particular  frequency  in  the  direction  shown  by  the  yellow  arrow.  “Ghost- 
elements”  corresponding  to  signal  delays  are  also  shown.  The  bottom  panel  shows  signals  arriving  at  four  of 
the  elements,  the  amounts  of  delay  on  those  elements,  the  resulting  delayed  signals  (or  the  signals  arriving  at 
the  ghost  elements),  and  their  sum.  In  this  case,  noise  arriving  from  a  direction  other  than  the  steer  angle  is 
added  out  of  phase  and  effectively  nulled.  This  interactive  simulation  is  also  capable  of  modeling  multi¬ 
aperture  and  other  array  geometries.  The  student  or  instructor  can  change  any  of  the  parameters  in  these 
displays,  in  order  to  investigate  beam  width  and  directivity  as  a  function  of  array  design  and  employment. 
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Figure  3  shows  a  view  from  the  interactive  modeling  facility  for  transmission  loss  in  the  ocean 
environment.  This  allows  a  visual  exploration  of  sound  propagation  paths  due  to  reflection  and  refraction.  A 
sound  speed  profile  (SSP)  is  displayed  on  the  left  of  the  display.  The  bottom  type,  SSP,  and  bottom  contour 
data  can  be  manually  entered  or  extracted  from  high-resolution  databases.  The  top  right  panel  shows  an 
example  full-field  plot  of  energy  loss,  with  the  bottom  panel  showing  transmission  loss  over  range  at  the 
sensor  depth  indicated  by  the  white  horizontal  line  in  the  top  panel.  The  user  can  simply  drag  the  depth  line 
to  update  the  transmission  loss  plot.  All  the  factors  that  affect  transmission  loss,  such  as  spreading, 
absorption  by  the  bottom,  and  scattering  at  the  bottom  and  surface,  are  modeled  and  contribute  to  the 
interactive  displays.  IMAT  includes  extensive  range-dependent  propagation  loss  models,  and  databases  of 
environmental  data  (such  as  sound  speed  or  bottom  absorption)  approved  by  the  Oceanographer  of  the  Navy. 
With  these  modules,  a  user  can  select  any  geographic  location  and  time  of  year,  extract,  view,  enter,  and/or 
modify  environmental  data,  specify  source  and  target  depths  and  frequency  of  interest,  and  then  investigate 
propagation  loss  as  a  function  of  depth,  distance  and  azimuth  from  either  a  sensor  or  threat. 
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Task  Analysis  and  development  of  Training  Objectives: 

The  model-based  scientific  visualizations  in  IMAT  have  also  enabled  a  new  approach  to  the  specification  of 
training  objectives.  The  analysis  of  complex  conceptual  tasks  for  training  and  educational  purposes  has  been 
a  central  problem  in  instmctional  psychology  for  at  least  the  past  50  years.  The  traditional  method  for 
analyzing  a  task,  as  described  in  most  military  training  development  guidance  documents  (e.g.  NAVEDTRA 
130  series  in  the  US  Navy),  is  to  identify  the  components  of  a  task  by  hierarchically  decomposing  it  into 
subtasks,  skills,  and  knowledge.  Training  is  then  based  on  these  units,  and  they  are  tested  mostly 
individually.  This  unfortunately  results  in  a  focus  on  low-level  detail  in  training,  so  that  students  are 
presented  materials  as  independent  topics  taught  in  a  serial  fashion  with  limited  cause-and-effect  explanation 
as  to  how  those  topics  interrelate.  This  approach  often  leads  to  instruction  in  which  students  read  and  listen 
to  lengthy  descriptions  of  complex  phenomena  and  memorize  large  amounts  of  factual  data  without  any 
contextual  reference.  These  methods  produce  graduates  who  can  answer  specific  factual  questions  based  on 
memorized  information  and  who  can  perform  procedures  but  cannot  apply  that  knowledge  in  operational 
situations.  For  example,  before  IMAT,  acoustic  operators  received  instruction  about  oceanography  and  the 
physics  of  sound  as  if  they  were  independent  of  a  target  submarine’s  operating  environment,  mission 
dependent  acoustic  characteristics,  and  the  displayed  frequency-based  information  resulting  from  sensor 
system  engineering  design.  Operators  trained  with  this  approach  could  recite  from  memory  specific  threat 
emitter  parameters,  but  did  not  necessarily  know  how  to  optimally  employ  a  sensor  system  against  that  threat 
under  specific  environmental  conditions.  These  operators  could  answer  knowledge-based  questions  and 
perform  procedures,  but  when  confronted  with  a  variety  of  real-world  situations,  they  were  not  able  to 
perform  adequately.  The  operational  problems  that  result  include  inefficient  threat  detection  and  ineffective 
environmental  analysis. 

These  sorts  of  performance  deficiencies  are  the  result  of  the  application  of  traditional  analysis  and 
instructional  methods  to  these  types  of  tasks.  As  Feltovich  et  al  (1991)  point  out: 

“...common  strategies  of  simplification.. .such  as  teaching  topics  in  isolation  from  related  ones 
(compartmentalizing  knowledge),  presenting  only  clear  instances  (and  not  the  many  pertinent 
exceptions),  and  requiring  only  reproductive  memory  in  assessment  are  often  in  conflict  with  the  realities 
of  advanced  learning — ^where  components  of  knowledge  are  fundamentally  interrelated,  where  context- 
dependent  exceptions  pervade,  and  where  the  ability  to  respond  flexibly  to  “messy”  application  situations 
is  required.” 

Further,  when  task  analysis  results  in  the  introductory  instruction  for  complex  interrelated  tasks  being  taught 
as  a  series  of  isolated  topics,  there  may  be  a  detrimental  effect  on  future  learning: 

“We  have  found  these  discrepancies  between  introductory  and  advanced  learning  often  result  in 
situations  where  the  groundwork  set  down  in  introductory  learning  actually  interferes  with  successful 
advanced  learning.” 

More  modem  conceptual  analysis  methods  take  a  different  approach;  they  focus  on  the  interrelationships  of 
concepts  in  a  technical  domain,  and  for  instmctional  design  purposes  they  attempt  to  analyze  the  thought 
processes  of  the  performer  during  performance  of  a  task.  However,  proponents  of  cognitive  task  analysis 
have  not  yet  had  much  success  at  developing  their  methods  so  that  they  can  be  routinely  applied  in  complex 
warfighting  tasks,  and  in  some  cases  this  approach  has  led  to  overly  narrow  characterizations  of  Navy 
training  requirements. 

The  IMAT  project  has  led  to  a  process  for  conducting  conceptual  analyses,  which  involves  the  following 
general  steps: 

a.  Define  the  most  complex  performance  problem  for  which  a  training  solution  is  required 

b.  Identify  and  refine  the  variables,  and  dimensions  along  which  they  vary,  necessary  to  model  the 
problem. 
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c.  Obtain  or  develop  mathematical  and/or  qualitative-process  models  which  relate  these 
variables/dimensions  and  specify  how  they  interact. 

d.  Design  an  interface  and  display  system  which  facilitates  understanding  of  the  variables  and  their 
relationships 

e.  Identify  problem  scenarios  (cases)  using  the  resulting  simulation. 

f  Validate  the  problem  scenarios  by  working  through  them  with  operators  and  tacticians. 

In  general,  the  process  of  constructing  and  validating  model-based  visualization  systems  identifies  the 
underlying  critical  variables,  their  relationships,  and  their  tactical  implications.  These  then  become  the 
enabling  concepts  and  tasks  in  the  analysis.  This  analytic  methodology  has  now  been  successfully  applied 
for  acoustic,  electromagnetic,  and  electro-optical  systems,  including  revealing  employment-training 
requirements  for  developmental  systems  still  in  test  and  evaluation. 


IMAT  Training  Techniques 

The  IMAT  effort  provided  a  unique  opportunity  to  integrate  and  jointly  evaluate  several  of  the  developing 
cognitive  techniques,  including  cognitive  modeling,  situated  learning,  elaborated  explanations,  and  graphical 
techniques  to  promote  visualization.  The  IMAT  effort  has  adopted  and  tested  several  modern  approaches  to 
complex  skill  instruction,  including  the  following: 

a.  Contextualized/ Anchored/Situated  Instruction:  Task  or  job  oriented  instruction  has  been  found  to  be 
more  effective  in  learning,  retention,  and  performance  than  topic  oriented  instmction  (Semb  &  Ellis,  1 994; 
Johnson,  1951;  Duffy  and  Jonassen,  1991;  Shoemaker,  1960;  Steinemann,  Harrigan,  &  VanMatre,  1967; 
Cognition  &  Technology  Group  at  Vanderbilt,  1990;  Collins,  Brown,  &  Newman,  1989).  IMAT  has 
employed  this  approach  in  basic  and  advanced  sensor  employment  and  mission  planning  courses.  (Czech,  C., 
Walker,  D.,  Tarker,  B.,  &  Ellis  J.A.,  1998). 

b.  Graphic  Displays/Interfaces  Illustrate  Cause  and  Effect  Relationships  and  Help  Concretize  Invisible 
Phenomena  and  Events:  Research  on  learning  from  text  has  shown  that  adding  pictures  or  graphics  aids 
learning  and  retention  if  they  supplement  the  text  in  some  meaningful  way  (Dwyer,  1972;  Gropper,  1966; 
Royer  &  Cable,  1976).  Levie  and  Lentz  (1982)  in  a  meta-analysis  of  illustrated  text  studies  concluded  that 
learning  and  retention  is  facilitated  by  illustrations,  if  the  illustrations  are  directly  related  to  the  text.  The 
IMAT  effort  has  shown  that  delivering  instruction  via  graphical  interfaces  to  conceptual  models  has  a  great 
effect  on  subsequent  performance,  both  for  apprentice  and  advanced  tactical  planning  tasks  (Wetzel-Smith, 
S.K.  &  C.  Czech.  (1996). 

c.  Elaborated  Explanations  of  Complex  Tasks  and  Phenomena:  Providing  students  with  elaborated 
explanations,  analogies,  etc.  about  how  and  why  systems,  events,  and  phenomena  are  structured  and  function 
should  facilitate  learning  and  retention.  Research  on  learning  skills  and  learning  from  text  has  shown  that 
elaborated  explanations  enhance  the  students’  mental  models  and  increase  retention  (Mayer,  1989;  Konoske 
&  Ellis,  1991;  Smith  &  Goodman,  1982;  Swezey,  Perez  &  Allen,  1991).  IMAT  has  shown  the  same  effects 
with  Navy  warfighting  tasks  (Ellis,  J.A.  Tarker,  B.,  Devlin,  S.E.  and  Wetzel-Smith,  S.K.  1997). 

d.  Instructional  Sequencing:  Mental  model  development  is  facilitated  by  teaching  students  to  reason 
about  events  and  phenomena  that  involve  several  interrelated  variables.  Earlier  research  on  sequencing 
showed  that  with  simplified  or  isolated  tasks,  different  sequences  of  instructional  events  made  little 
difference.  However,  recent  research  and  theory  suggests  that  for  complex  tasks,  sequencing  strategies  may 
have  significant  effects,  and  these  are  being  observed  in  IMAT  courses  (Czech,  C.,  Walker,  D.,  Tarker,  B.,  & 
Ellis  J.A.,  1998). 
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In  each  of  these  areas,  before  IMAT,  little  experimental  work  had  been  done  on  the  extent  to  which  the 
findings  are  generalizable  to  instruction  delivered  using  simulation-  and  graphical-interface-based  training 
technologies.  Furthermore,  there  are  almost  no  larger  efforts  that  integrate  all  these  approaches  into  an 
overall  strategy.  IMAT  has  proven  the  notion  that  a  combination  of  these  approaches  will  offer  a  potent 
learning  environment  for  promoting  acquisition  of  the  kinds  of  complex  skills  involved  in  sensor-system 
operation  and  tactical  planning. 


Conceptual  Training  during  Exercise  Planning  /  Execution  /  Reconstruction 

In  tactical  employment  training,  tasks  involving  planning  and  mission  execution  for  any  particular  operating 
environment  require  that  a  plaimer  understands  how  best  to  optimize  the  mix  of  sensor  capabilities  to  detect 
and  prosecute  the  threat.  At  the  platform  level,  planners  will  have  to  understand  which  systems  and  system 
settings  to  select  to  best  detect  and  prosecute  an  attack  while  accurately  estimating  likelihood  of 
counterdetection  and  potential  vulnerability.  The  tactician  will  need  to  predict  the  environmental  effects  on 
each  of  the  sensors  to  be  able  to  effectively  plan  and  execute  the  mission.  Previously,  the  training  available 
to  tactical  planners  tended  to  familiarize  them  with  available  environmental  products,  but  did  little  to  teach 
them  how  to  apply  the  information  in  tactical  planning  and  mission  execution.  IMAT  training  now  allows 
tacticians  to  practice  this  sort  of  plaiming  and  mission  execution,  and  IMAT  systems  can  be  used  for  timely 
reconstruction  of  exercises. 

Over  the  past  several  years,  the  IMAT  approach  to  tactical  training  has  been  explored  in  connection  with  at- 
sea  exercises.  During  an  exercise,  IMAT  researchers  work  on  board  with  operators  and  command  persoimel 
to  provide  additional  training  in  tactical  use  of  the  ocean  in  the  context  of  the  ship’s  performance  during  each 
watch  period.  Results  typically  indicate  important  performance  improvements  in  the  ship’s  sonar  operations. 
Moreover,  these  improvements  are  retained  during  subsequent  readiness  evaluations. 

This  sort  of  exercise-based  training  has  been  repeated  with  dozens  of  submarines  and  surface  ships.  .  To 
support  these  exercises,  IMAT  researchers  often  spend  many  weeks  developing  mission  analyses,  including 
oceanographic  workups,  sensor  performance  predictions,  and  counterdetection  assessments,  as  well  as 
displays  and  visualizations  to  deliver  the  training.  The  development  of  deployable  mission  analysis  and 
reconstruction  training  will  require  developing  authoring  and  visualization  tools  which  can  support  much 
more  rapid  scenario  (re)constmction  and  display.  The  experience  gained  in  these  efforts  has  led  to  further 
definition  of  the  requirements  for  deployable  mission  analysis  and  reconstruction  tools 

For  interactive  mission  analysis  training,  novel  methods  of  curriculum  design  are  needed.  IMAT  training 
involves  extensive  use  of  case-  or  situation-based  analysis/reconstruction  training  usable  in  several  types  of 
training  scenarios: 

•  Provide  a  conceptual  overview  which  gives  an  integrated  expert-model  based  approach  to 
understanding  the  variables  affecting  sonar  tactics. 

•  Run  pre-built  training  episodes  which  contain  teaching  points  /  instructional  strategies  for 
developing  tactical  skill. 

•  Use  analysis  capabilities  of  IMAT  visualization  systems  to  explore  and  critique  sonar  tactical 
planning  for  at-sea  exercises. 

•  Use  analysis  capabilities  of  the  visualization  systems  during  exercise  execution  to  explore  what- 
if  options  for  ownship  and  target(s).  Use  analysis  /  reconstruct  capabilities  to  match  predicted 
conditions  with  actual,  then  modify  plans  /  projections. 

•  Use  analysis  /  reconstruct  capabilities  for  post-exercise  sensor  /  tactics  training  assessment  to 
provide  “lessons-learned”  reinforcement. 
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Most  recently,  these  techniques  have  been  extended  to  the  force  level,  wherein  several  ships  and  aircraft 
conduct  joint  ASW  operations.  These  forces  are  under  the  command  of  a  senior  tactician  who,  with  his  staff, 
prepares  coordinated  plans  for  the  ASW  problem,  then  monitors  the  execution  of  the  plans,  and  ultimately  is 
responsible  for  reconstruction  and  feedback  to  the  units  and  personnel  involved.  To  support  ASW  at  the 
force  level,  IMAT  researchers  have  built  visualization  systems  that  can  be  networked  among  the  platforms 
involved  to  support  collaborative  training  and  mission  execution.  The  US  Navy  has  established  IMAT  Fleet 
Training  Teams  that  deliver  training  at-sea  at  the  platform  and  force  levels. 


Evaluations  of  Training  Effectiveness 

Measures  of  Effectiveness  for  Visualization-based  training  systems  involve  several  different  criteria. 
Training  and  performance  aiding  systems  must  first  capture  the  complexity  of  real-world  operations.  This  is 
assessed  through  developmental  test  and  sensitivity  analyses  for  performance  prediction  systems  and  through 
continual  build-test-build  cycles  conducted  with  operational  users  of  the  systems  under  development.  (1) 
Performance  prediction  systems  must  properly  implement  validated  physics  models  and  approved  databases 
of  input  parameters.  (2)  These  systems  must  properly  model  all  major  physical  phenomena  known  to  affect 
sensor  /  platform  performance  in  real-world  operations.  (3)  Performance  prediction  and  visualization 
systems  must  by  usable  by  experienced  operators  and  tacticians  so  as  to  provide  support  through  major 
phases  of  their  tactical  tasks,  including  planning,  search,  and  prosecution.  They  should  add  little  additional 
complexity  to  operator  and  tactician  tasks.  IMAT  systems  are  subject  to  these  kinds  of  criteria — they  have 
been  independently  tested,  and  adopted  for  use  as  fleet-approved  tactical  decision  aids  by  the  US  and  other 
Navies. 

Training  improvements  are  assessed  through  pretest-posttest  and  training-vs-control  evaluations  on  tests 
requiring  knowledge  and  skill  application  on  scenario-based  tactical  reasoning  problems.  IMAT 
visualization-based  training  consistently  result  in  additional  operator  /  tactician  knowledge  and  skill. 
Evaluations  of  training  effectiveness  in  ASW  schools  indicate  that  IMAT  is  among  the  most  successful 
training  technologies  ever  introduced  in  the  US  Navy  (Committee  on  Technology  for  Future  Naval  Forces, 
1997): 

•  IMAT  students  outperform  students  in  conventional  instruction,  and  in  many  cases  score  higher  than 
qualified  fleet  personnel  with  3  to  10  years  experience.  Evaluations  consistently  show  gains  of  two  to 
three  standard  deviations  on  comprehension,  reasoning,  and  problem  solving  tasks.  Overall,  the  IMAT 
approach  is  much  more  effective  than  conventional  lecture  instruction,  or  new  technologies  such  as 
interactive  video  or  computer-based  training. 

•  Instructors  report  that  IMAT  increases  their  ability  to  teach  difficult  topics,  respond  to  student  questions, 
and  reinforce  critical  principles. 

•  IMAT  students  score  higher  on  attitude  scales  measuring  attention,  relevance,  confidence,  and 
satisfaction  than  students  in  standard  Navy  classrooms  or  students  in  specially  designed  individualized 
computer  based  training. 

Finally,  improved  training  should  result  in  observable  improvements  in  operator  and  tactician  performance 
during  exercises  and  operations,  for  example  in  improvements  in  the  quality  and  timeliness  of  tactical  plans 
and  decisions  made  during  tactical  execution.  Measures  include:  (1)  better  utilization  of  platforms  (e.g.  less 
overlap  in  sensor  coverage),  (2)  deeper  user  examination  of  alternative  courses  of  action  during  planning  and 
execution,  (3)  improved  availability  and  processing  of  information  necessary  to  support  planning  and 
execution  monitoring  (e.g.  minimize  time  spent  collecting  information  while  maximize  time  available  for 
information  analysis). 
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Ultimately  of  course,  improved  training  and  performance  support  systems  should  result  in  measurable 
improvements  in  military  capability.  Measures  should  reveal  (1)  increased  detection  /  engagement  ranges, 
(2)  increased  search  rates  or  area  coverage,  (3)  more  rapid  localization  and  classification,  (4)  reduction  in 
prosecution  of  false  contacts,  (5)  reduction  in  counter-detection  and  counter-attack  vulnerability,  and  (6) 
increased  tactical  control  or  advantage.  All  of  these  measures  are  highly  dependent  on  the  boundary 
conditions  for  tactical  exercises  and  operations  -  they  can  vary  widely  depending  on  any  or  all  of  the 
variables  (e.g.,  threat  parameters,  environmental  conditions,  sensor  employment,  target  behavior  and  tactics, 
ambient  noise,  etc.)  that  normally  complicate  military  operations.  IMAT  researchers  and  training-team 
instructors  have  provided  decision-aiding  systems  and  advanced  training  to  individual  ships  and  to 
commanders  and  command  staffs  for  battle  groups.  Independent  evaluations  of  battle-group  performance 
reveal  improvements  in  all  these  measures. 


Conclusion 

The  IMAT  program  is  providing  training  and  performance  support  systems  designed  to  make  difficult 
scientific  and  technical  concepts  comprehensible  to  the  operational  users  of  advanced  sensor  systems.  The 
program  is  (1)  developing  systems  which  integrate  computer  models  of  physical  phenomena  with  scientific 
visualization  technologies  to  demonstrate  the  interactive  relationships  of  threat,  environment,  and  sensor  for 
operator  training,  and  interactions  of  multiple  sensor  systems  for  tactician  training;  (2)  developing  training 
and  performance  support  systems  using  modeling  and  visualization  technologies;  (3)  integrating  curricula  to 
provide  training  on  high-level  sensor  operation  and  tactical  planning  skills;  and  (4)  developing  modeling  and 
visualization  tools  for  use  at  sea  both  for  training  and  as  tactical  decision  aids. 

The  IMAT  vision  is  to  integrate  training,  operational  preparation,  tactical  execution,  and  post-mission 
analysis  into  a  seamless  support  system  for  developing  and  maintaining  mission-related  critical  skills.  In 
many  ways,  IMAT  is  a  prototype  for  future  human  performance  support  systems  that  transcend  traditional 
shore  school  and  course  structures  to  span  career-long  skill  development  from  apprentice  to  master  levels, 
across  missions,  platforms,  and  communities. 
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